Skip to content

feat(manipulation): add VAMP planner backend#2522

Open
TomCC7 wants to merge 26 commits into
mainfrom
cc/feat/vamp
Open

feat(manipulation): add VAMP planner backend#2522
TomCC7 wants to merge 26 commits into
mainfrom
cc/feat/vamp

Conversation

@TomCC7

@TomCC7 TomCC7 commented Jun 17, 2026

Copy link
Copy Markdown
Member

Problem

DimOS manipulation planning is currently centered on the Drake world plus RRT-Connect stack. We need an optional VAMP backend for fast joint-space planning, plus a mock-control Franka Panda target for testing and future benchmarking against an official VAMP robot.

Closes N/A

Solution

  • Add typed nested world/planner config variants for Drake/RRT-Connect and VAMP.
  • Add strict stack validation so VAMP world and VAMP planner are paired explicitly, while preserving Drake defaults.
  • Add lazy optional VAMP integration:
    • VampWorld loads official or user-prepared artifacts, handles native validity/FK/environment conversion, and reports unsupported capabilities clearly.
    • VampPlanner handles VAMP algorithm selection, joint-space planning, optional simplification, and path validation.
  • Add a mock-control Franka Panda catalog, LFS-backed robot description asset, and panda-coordinator blueprint for planning tests/benchmarks.
  • Add manipulation planning docs, OpenSpec artifacts, and tests covering config parsing, pairing validation, fake VAMP modules, Panda catalog conversion, and blueprint registration.

Design notes:

  • VAMP is intentionally world + planner only, not a kinematics backend.
  • VAMP Jacobian support is not faked; unsupported capabilities raise clear errors.
  • vamp-planner is optional and lazy-imported behind the new vamp extra.
  • This PR also includes the OpenSpec schema/init files used to capture the change plan.

How to Test

uv run pytest dimos/robot/manipulators/franka/test_blueprints.py dimos/robot/catalog/test_franka.py dimos/manipulation/test_manipulation_unit.py dimos/manipulation/planning/vamp/test_vamp_backend.py -q
CI=1 uv run pytest dimos/robot/test_all_blueprints_generation.py -q
uv run doclinks --root . --check docs/usage/manipulation_planning.md docs/usage/README.md
openspec validate add-vamp-planning-backend

Manual mock Panda/VAMP smoke path:

uv run --extra manipulation --extra vamp dimos run panda-coordinator --daemon \
  -o manipulationmodule.world.backend=vamp \
  -o manipulationmodule.world.artifact.mode=official \
  -o manipulationmodule.world.artifact.robot=panda \
  -o manipulationmodule.planner.backend=vamp \
  -o manipulationmodule.planner.algorithm=rrtc

Contributor License Agreement

  • I have read and approved the CLA.

@TomCC7 TomCC7 changed the title feat(manipulation): add VAMP planner backend WIP: feat(manipulation): add VAMP planner backend Jun 17, 2026
@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.42105% with 126 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/manipulation/planning/world/vamp_world.py 63.41% 61 Missing and 14 partials ⚠️
dimos/manipulation/planning/vamp/loader.py 62.50% 17 Missing and 7 partials ⚠️
dimos/manipulation/planning/factory.py 41.17% 15 Missing and 5 partials ⚠️
...os/manipulation/planning/vamp/test_vamp_backend.py 98.63% 3 Missing ⚠️
dimos/manipulation/manipulation_module.py 93.33% 2 Missing ⚠️
...mos/manipulation/planning/planners/vamp_planner.py 86.66% 1 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main    #2522      +/-   ##
==========================================
+ Coverage   69.97%   70.10%   +0.13%     
==========================================
  Files         842      853      +11     
  Lines       74420    75132     +712     
  Branches     6668     6717      +49     
==========================================
+ Hits        52077    52673     +596     
- Misses      20636    20728      +92     
- Partials     1707     1731      +24     
Flag Coverage Δ
OS-ubuntu-24.04-arm 63.99% <83.28%> (+0.08%) ⬆️
OS-ubuntu-latest 64.81% <83.28%> (+0.07%) ⬆️
Py-3.10 64.81% <83.28%> (+0.07%) ⬆️
Py-3.11 64.81% <83.28%> (+0.07%) ⬆️
Py-3.12 64.81% <83.28%> (+0.07%) ⬆️
Py-3.13 64.81% <83.28%> (+0.07%) ⬆️
Py-3.14 64.82% <83.28%> (+0.07%) ⬆️
Py-3.14t 64.81% <83.28%> (+0.08%) ⬆️
SelfHosted-Large 30.46% <37.10%> (+0.18%) ⬆️
SelfHosted-Linux 38.22% <38.28%> (+<0.01%) ⬆️
SelfHosted-macOS 37.01% <38.28%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...mos/manipulation/planning/monitor/world_monitor.py 52.39% <100.00%> (+0.35%) ⬆️
dimos/manipulation/planning/planners/config.py 100.00% <100.00%> (ø)
dimos/manipulation/planning/vamp/errors.py 100.00% <100.00%> (ø)
dimos/manipulation/planning/world/config.py 100.00% <100.00%> (ø)
dimos/manipulation/test_manipulation_unit.py 100.00% <100.00%> (ø)
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
dimos/robot/catalog/franka.py 100.00% <100.00%> (ø)
dimos/robot/catalog/test_franka.py 100.00% <100.00%> (ø)
dimos/robot/config.py 74.24% <ø> (ø)
dimos/robot/manipulators/franka/blueprints.py 100.00% <100.00%> (ø)
... and 7 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread dimos/manipulation/planning/planners/config.py Outdated
Comment thread dimos/manipulation/planning/planners/vamp_planner.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/vamp/loader.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/manipulation/planning/world/vamp_world.py Outdated
Comment thread dimos/robot/config.py Outdated
Comment thread dimos/robot/catalog/franka.py Outdated
Comment thread dimos/utils/data.py Outdated
Comment thread dimos/utils/data.py Outdated
@TomCC7 TomCC7 changed the title WIP: feat(manipulation): add VAMP planner backend feat(manipulation): add VAMP planner backend Jun 19, 2026
@TomCC7 TomCC7 marked this pull request as ready for review June 19, 2026 08:11
@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds an optional VAMP planner backend for joint-space motion planning, alongside a mock-control Franka Panda catalog entry and panda-coordinator blueprint for testing. The implementation uses typed Pydantic config variants with discriminated unions for world/planner selection, strict stack validation to enforce VAMP world ↔ VAMP planner pairing, and lazy imports so the vamp-planner extra is only required when actually configured.

  • VAMP backend: VampWorld wraps VAMP's native validity checking, FK, and environment APIs; VampPlanner is a thin adapter that delegates to VampWorld.plan_joint_path, which handles algorithm selection, optional simplification, and path validation.
  • Loader: Official VAMP robot modules (panda, ur5, etc.) are imported directly; custom artifacts can be loaded from a directory or file path with the previous sys.path mutation issue now fixed via try/finally cleanup.
  • Franka Panda catalog: Adds LFS-backed URDF/SRDF assets, explicit joint names and home config, and a panda-coordinator mock-control blueprint pairing ManipulationModule with ControlCoordinator.

Confidence Score: 4/5

Safe to merge with one fix in loader.py's file-based custom artifact path.

The file-based custom artifact loader adds the module to sys.modules before exec_module completes. If loading fails, the broken module object is permanently cached under the file's stem name. Any subsequent attempt to load that same stem — even after fixing the artifact — silently returns the broken object for the remainder of the process. This affects anyone using the CustomVampArtifactConfig file path variant; the official robot path is unaffected. The rest of the implementation is well-structured: lazy imports are correctly deferred, the previous truncation and sys.path cleanup issues are resolved, and the typed config/validation stack is clean.

dimos/manipulation/planning/vamp/loader.py — the file-based custom artifact branch needs a try/except around exec_module to remove the broken entry from sys.modules on failure.

Important Files Changed

Filename Overview
dimos/manipulation/planning/vamp/loader.py Loads official and custom VAMP robot modules; directory-based custom artifacts now clean up sys.path via try/finally (previous issue resolved), but the file-based path adds the module to sys.modules before exec_module completes — a failed or stale load permanently poisons that module name in the process.
dimos/manipulation/planning/world/vamp_world.py VAMP WorldSpec adapter; truncation issue in plan_joint_path addressed (start/goal now go through _joint_state_for_robot_order); cylinder obstacles are silently approximated as capsules without user-visible warning.
dimos/manipulation/planning/factory.py Factory now correctly defers VampWorld/VampPlanner imports inside the factory functions; only config models (no VAMP binary) are imported at module level, preserving the lazy-load contract.
dimos/manipulation/planning/planners/vamp_planner.py Thin adapter that delegates to VampWorld.plan_joint_path; correctly validates world type and forwards config.
dimos/manipulation/manipulation_module.py Integrates typed world/planner/kinematics configs with stack validation; deprecation shims for legacy flat fields are clean.
dimos/robot/catalog/franka.py New Franka Panda catalog entry with LFS-backed URDF/SRDF; explicit joint names, home joints, and package paths look correct.

Reviews (2): Last reviewed commit: "fix(manipulation): address Greptile VAMP..." | Re-trigger Greptile

Comment thread dimos/manipulation/planning/world/vamp_world.py
Comment thread dimos/manipulation/planning/factory.py Outdated
Comment thread dimos/manipulation/planning/vamp/loader.py Outdated
Comment thread dimos/manipulation/planning/vamp/loader.py
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jun 19, 2026
Comment thread pyproject.toml
vamp = [
# Optional VAMP planning backend. Keep out of broad extras to avoid forcing
# native planner bindings into non-VAMP manipulation environments.
"vamp-planner @ git+https://github.com/TomCC7/vamp.git@2b59552179c2af473c59c8617519e4663497fbe6",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You cannot add git urls in dependencies because we can't publish that to PyPI. PyPI only allows dependencies for other packages on PyPI.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should publish this package as well if we decide to support this planner, @mustafab0 's call

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then I guess the same thing for roboplan, but for that I contacted the maintainer: open-planning/roboplan#247

@@ -0,0 +1,53 @@
# Copyright 2025-2026 Dimensional Inc.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this file and add another bullet point to the skill to not add tests for blueprint values (blueprints are configs, not functionality).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants